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AUTOMATIC TRACK GENERATION 
Field of the Invention 

The current invention relates to video editing and in particular to multiple track 

editing. 

5 Background 

Video and audio productions often comprise multiple independent tracks. For 
example, a typical motion picture comprises a single video track, and two or more audio 
tracks. During the creation process for such productions, technicians frequently deal with 
more tracks than will be used in the final production. In the editing phase of the creation, 

10 editors separately edit these individual tracks to produce desired outcomes, but they 
remain mindful of the combined effect. 

In the course of editing video and audio productions it is often important that one 
track stays synchronised with another. For example, the video of a actors lip movements 
is commonly closely synchronised with the relevant audio relating the actors words. As 

15 time related aspects of a "source" video track relating to the actors lips are edited 
(deleted, inserted or moved etc.) the related (synchronised) "dependent" audio is 
correspondingly edited in cases where is desired to maintain this synchronised 
relationship. 

In many computer-based non-linear editors the editing systems display a 
20 visualisation, often referred to as a timeline, of the multiple tracks being edited. This 
mechanism enables a user to observe a loss of synchronisation between tracks and to 
manually restore the relationship by editing the dependant tracks. This is however a 
tedious and error prone approach, since the user needs to continually adjust one or other 
track to take account of relative changes between the tracks. 



210303 



629349.DOC 



- 2 - 

Some systems allow synchronisation between different tracks to be maintained 
by Mocking' together the separate tracks. This enables the system to automatically 
preserve a synchronised timing relationship between the tracks. For some track types this 
locked type of synchronisation is, however, not desirable. Thus, for example, tracks such 
5 as background music or overlay video while dependent on a source video track, should be 
re-generated whenever the source track changes, rather than precisely reflecting the 
changes to the source track. 

For example, consider the required changes to a dependent audio background 
music track, and a dependent audio dialog track when an associated source video track is 
10 edited. If some content is deleted between time Tj and time T 2 from the source video 
track, the corresponding content needs to be deleted from the dependent dialog track. 
This corresponding content should not, however, be deleted between these time instants 
in the dependent background music track, since to do so would cause a discontinuity in 
the background music track. A more palatable solution is for the duration of the 
15 background music track to be shortened by T 2 - T|. Similarly, the dependent background 
music track may have its duration increased in response to content being inserted in the 
source video track, possibly requiring looping if the total duration of the dependent 
background music track is not sufficient. 

Mechanisms for providing automatic synchronisation in current systems are 
20 complex, for one or more of the following reasons: 

• If synchronisation of a dependent track, with reference to a region of a source 

track, is possible at all, it is generally necessary to define both a beginning and 

end position of that region of the source track to which the dependent track is to 

be synchronised. 
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• The aforementioned regions of the source track cannot simply be redefined while 
preserving properties of the generated dependent track. For example the 
definition of the media to be used for the generated dependent track is not 
preserved. In other words, the region of the source track to which the dependent 
track is synchronised cannot be redefined without removing the synchronisation 
entirely, and defining it anew. This redefinition requires the attributes of the 
synchronisation, including which media is to be used in the generation, of the 
relevant portion of the dependent track, to be redefined. 

• Users typically need to consider how the synchronisation method is operating, 
and need to incorporate this factor in their editing rather than being able to focus 
on the key task of editing tracks. 

There are also problems in integrating automatic synchronisation in systems that 
provide manual post-editing of automatically generated productions. Typically these 
systems are designed for users with limited experience of computers and/or video editing. 
Such users may simply desire the convenience of a system that is simple to use. In these 
systems, the automatic editing rules have been designed by professionals to ensure a good 
result, but the systems lack robust mechanisms to ensure that the intent of the professional 
is preserved throughout the editing process. 

Summary of the Invention 

It is an object of the present invention to substantially overcome, or at least 
ameliorate, one or more disadvantages of existing arrangements. 

Disclosed are arrangements which seek to address the above problems by loosely 
synchronizing, to a source media item, a dependent media item, so that the dependent 
media item is coupled to the source media item, the coupling being subject to moderation 
by other media items in the neighbourhood of the source media item. 
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According to a first aspect of the present invention, there is provided a method of 
generating, in relation to a source placeholder, a dependent media item, said method 
comprising the steps of: 

determining, from an EDL element associated with the source placeholder, a 
5 plurality of track control attributes; and 

establishing the dependent media item according to the track control attributes. 
Other aspects of the invention are also disclosed. 

Brief Description of the Drawings 
One or more embodiments of the present invention will now be described with 
10 reference to the drawings, in which: 

Fig. 1 is a functional block diagram of a general-purpose computer upon which 
the described methods for automatic track generation can be practiced; 

Fig. 2 shows relationships between a source Edit Display List (EDL), an 
associated source media track, and an associated dependent media item; 
15 Fig. A shows track control attributes aggregated within an EDL element of a 

source media item; 

Fig. B shows successive EDL elements for a track, the EDL elements containing 
media item information and track control attributes with values shown for a number of 
attributes; 

20 Fig. C shows an alternate data structure for EDL elements of the source track in 

question, the EDL elements containing the same media item information and track 
attributes as Fig. B; 

Figs. Da and Db show visual representations of source tracks and associated 
dependent overlay tracks generated from the EDL of the source tracks; 



210303 



629349.DOC 



m 

-5- 

Fig. E is a tabular representation of the attributes associated with Figs Da and 

Db; 

Fig. F shows a process for generating a dependent track from a source track; and 
Fig G shows an example of a Graphical User Interface (GUI) allowing user 
5 interaction to the automatic track generation method. 

Detailed Description including Best Mode 
It is to be noted that the discussions contained in the "Background" section 
relating to prior art arrangements relate to discussions of documents or devices which 
form public knowledge through their respective publication and/or use. Such should not 
10 be interpreted as a representation that such documents or devices in any way form part of 
the common general knowledge in the art. 

Where reference is made in any one or more of the accompanying drawings to 
steps and/or features, which have the same reference numerals, those steps and/or features 
have for the purposes of this description the same function(s) or operation(s), unless the 
15 contrary intention appears. 

Some portions of the description that follows are explicitly or implicitly 
presented in terms of algorithms and symbolic representations of operations on data 
within a computer memory. These algorithmic descriptions and representations are the 
means used by those skilled in the data processing arts to most effectively convey the 
20 substance of their work to others skilled in the art. An algorithm is here, and generally, 
conceived to be a self-consistent sequence of steps leading to a desired result. The steps 
are those requiring physical manipulations of physical quantities. Usually, though not 
necessarily, these quantities take the form of electrical or magnetic signals capable of 
being stored, transferred, combined, compared, and otherwise manipulated. It has proven 



210303 



629349.DOC 



-6- 

convenient at times, principally for reasons of common usage, to refer to these signals as 
bits, values, elements, symbols, characters, terms, numbers, or the like. 

It should be bome in mind, however, that the above and similar terms are to be 
associated with the appropriate physical quantities and are merely convenient labels 
5 applied to these quantities. Unless specifically stated otherwise, and as apparent from the 
following, it will be appreciated that throughout the present specification, discussions 
utilizing terms such as "scanning", "calculating", "determining", "replacing", 
"generating" "initializing**, "outputting", or the like, refer to the action and processes of a 
computer system, or similar electronic device, that manipulates and transforms data 
10 represented as physical (electronic) quantities within the registers and memories of the 
computer system into other data similarly represented as physical quantities within the 
computer system memories or registers or other such information storage, transmission or 
display devices. 

The present specification also discloses apparatus for performing the operations 
15 of the methods. Such apparatus may be specially constructed for the required purposes, 

or may comprise a general purpose computer or other device selectively activated or 

reconfigured by a computer program stored in the computer. The algorithms and displays 

presented herein are not inherently related to any particular computer or other apparatus. 

Various general-purpose machines may be used with programs in accordance with the 
20 teachings herein. Alternatively, the construction of more specialized apparatus to 

perform the required method steps may be appropriate. The structure of a conventional 

general purpose computer will appear from the description below. 

In addition, the present specification also discloses a computer readable medium 

comprising a computer program for performing the operations of the methods. The 
25 computer readable medium is taken herein to include any transmission medium for 
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communicating the computer program between a source and a designation. The 
transmission medium may include storage devices such as magnetic or optical disks, 
memory chips, or other storage devices suitable for interfacing with a general-purpose 
computer. The transmission medium may also include a hard-wired medium such as 
5 exemplified in the Internet system, or wireless medium such as exemplified in the GSM 
mobile telephone system. The computer program is not intended to be limited to any 
particular programming language and implementation thereof. It will be appreciated that 
a variety of programming languages and coding thereof may be used to implement the 
teachings of the disclosure contained herein. 

10 Fig. 1 shows how the method of automatic track generation particularly lends 

itself to implementation on a general-purpose computer system 100, such as that shown in 
Fig* 1 wherein the process of Fig. F may be implemented as software, such as an 
application program executing within the computer system 100. In particular, the steps of 
method of automatic track generation are effected by instructions in the software that are 

15 carried out by the computer. The instructions may be formed as one or more code 
modules, each for performing one or more particular tasks. The software may also be 
divided into two separate parts, in which a first part performs the automatic track 
generation methods and a second part manages a user interface between the first part and 
the user. The software may be stored in a computer readable medium, including the 

20 storage devices described below, for example. The software is loaded into the computer 
from the computer readable medium, and then executed by the computer. A computer 
readable medium having such software or computer program recorded on it is a computer 
program product. The use of the computer program product in the computer preferably 
effects an advantageous apparatus for automatic track generation. 



210303 



629349.DOC 



-8- 

The computer system 100 comprises a computer module 101, input devices such 
as a keyboard 102 and mouse 103, output devices including a printer 115, a display 
device 1 14 and loudspeakers 1 1 7. A Modulator-Demodulator (Modem) transceiver 
device 116 is used by the computer module 101 for communicating to and from a 
5 communications network 120, for example connectable via a telephone line 121 or other 
functional medium. The modem 1 16 can be used to obtain access to the Internet, and 
other network systems, such as a Local Area Network (LAN) or a Wide Area Network 
(WAN), and may be incorporated into the computer module 101 in some 
implementations. 

10 The computer module 101 typically includes at least one processor unit 105, and 

a memory unit 106, for example formed from semiconductor random access memory 
(RAM) and read only memory (ROM). The module 101 also includes an number of 
input/output (I/O) interfaces including an audio-video interface 107 that couples to the 
video display 114 and loudspeakers 117, an I/O interface 113 for the keyboard 102 and 

15 mouse 103 and optionally a joystick (not illustrated), and an interface 108 for the 
modem 116 and printer 115. In some implementations, the modem 1116 may be 
incorporated within the computer module 101, for example within the interface 108. A 
storage device 109 is provided and typically includes a hard disk drive 110 and a floppy 
disk drive 111. A magnetic tape drive (not illustrated) may also be used. A CD-ROM 

20 drive 112 is typically provided as a non-volatile source of data. The components 105 
to 1 13 of the computer module 101, typically communicate via an interconnected bus 104 
and in a manner that results in a conventional mode of operation of the computer 
system 100 known to those in the relevant art. Examples of computers on which the 
described arrangements can be practised include IBM-PC's and compatibles, Sun 

25 Sparcstations or like computer systems evolved therefrom. 
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Typically, the application program for automatic track generation, as well as the 
media item files associated with source and dependent media tracks, are resident on the 
hard disk drive 110 and read and controlled in its execution by the processor 105. 
Intermediate storage of the program and any data fetched from the computer memory 109 
or the network 120 may be accomplished using the semiconductor memory 106, possibly 
in concert with the hard disk drive 110. In some instances, the application program for 
automatic track generation may be supplied to the user encoded on a CD-ROM or floppy 
disk and read via the corresponding drive 1 12 or 1 1 1, or alternatively may be read by the 
user from the network 120 via the modem device 116. Still further, the software for 
automatic track generation can also be loaded into the computer system 100 from other 
computer readable media. The term "computer readable medium" as used herein refers to 
any storage or transmission medium that participates in providing instructions and/or data 
to the computer system 100 for execution and/or processing. Examples of storage media 
include floppy disks, magnetic tape, CD-ROM, a hard disk drive, a ROM or integrated 
circuit, a magneto-optical disk, or a computer readable card such as a PCMCIA card and 
the like, whether or not such devices are internal or external of the computer module 101. 
Examples of transmission media include radio or infra-red transmission channels as well 
as a network connection to another computer or networked device, and the Internet or 
Intranets including e-mail transmissions and information recorded on Websites and the 
like. 

The method of automatic track generation may alternatively be implemented in 
dedicated hardware such as one or more integrated circuits performing the functions or 
sub functions of automatic track generation. Such dedicated hardware may include 
graphic processors, digital signal processors, or one or more microprocessors and 
associated memories. 
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The disclosed arrangement for automatic track generation makes use of 
particular data structures to represent the relationship between media tracks. In these data 
structures, attributes controlling the generation of a dependent track are optionally 
indirectly associated with media elements of a source track. A media element, when 
considering a video source track, typically comprises a video scene. Alternately, the 
media element could be based upon a video frame. 

Fig. 2 shows an EDL 902. The EDL 902 is composed of EDL elements 901, 
907. Each EDL element 901 is associated with a media item 904 in a source media track 
906. The source media track 906 comprises a sequence of media items 904, 908. An 
individual media item 909 comprises video frames 905. Each media item 904 in the 
source media track 906 has, as depicted by an arrow 903, a corresponding EDL element 
901 in the EDL 902. 

Fig. 2 also shows a dependent media item 91 1 that is "loosely synchronised", as 
depicted by a dashed arrow 910, with one or more of the media items 904, 908 in the 
source track 906. As previously noted, a source video item that is focussed on an actors 
lips needs to be tightly synchronised with a corresponding dependent audio item 
containing the actor's words. However, the dependent media item 911 in Fig- 2 is less 
tightly, i.e. more "loosely" coupled. This loose coupling is appropriate for dependent 
tracks such as a dependent background audio track that, while associated with a source 
video clip, does not require tight coupling as is required in the case of an actor speaking. 
A dependent track is typically made up of a sequence of dependent media items 91 1, each 
media item 91 1 being generated by track control attributes in a corresponding sequence of 
EDL elements. 

From an implementation perspective, the EDL element 901 is associated, as 
depicted by the arrow 903, with the media item 904. According to a first arrangement 
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(also referred to as an "aggregated" arrangement as will be described with reference to 
Fig. B), a sequence of the EDL elements 901, 907 contain track control attributes 912 
which establish, as depicted by an arrow 913, the synchronisation between the dependent 
media item 91 1 and the source track 906. 
5 According to the aggregated arrangement all the EDL elements in the source 

EDL 902 have track control attributes as depicted by a reference numeral 914. These 
track control attributes in the EDL elements 901, 907 effect the behaviour of the 
dependent media item 91 1. In a second arrangement (also referred to as a "decoupled" 
arrangement as will be described with reference to Fig. C), some of the track control 

10 attributes are incorporated into distinct EDL elements (referred to as track control 
elements) in the EDL of the source media track. In the decoupled arrangement some of 
the track control attributes are not included within the EDL elements for the media items. 

Returning to Fig. 2, the dependent media item 91 1 consists of media content that 
does not need to be closely synchronised with the source track 906, Often only the start 

15 time and duration of the dependent media element 91 1 need to be synchronised to the 
source track 906. One example of a dependent track is a video overlay track, and 
dependent tracks will alternately be referred to as overlays in this description. This is an 
example only of a dependent track, however, and dependent tracks other than overlays 
can also be generated using the disclosed method. Examples of source tracks include 

20 unprocessed (raw) video and dialog audio tracks. 

The track control attributes associated with the EDL elements 901, ... , 907 
comprise (a) an attribute identifying the media from which the dependent media item 91 1 
is to be generated and (b) attributes that serve to control commencement and termination 
of the dependent media item 91 1. The track control attributes (912, 914) of the source 

25 track 906 thus control (c) when the generation of the dependent media item 91 1 should 
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begin, (d) where the dependent media item 91 1 should end or (e) when new media should 
be used for a dependent media item, and (f) whether generation of the dependent media 
item 91 1 should be temporarily suspended. 



One example of these track control attributes is set out in Table 1 : 



Name 


Type 


Description 


Apply 


Boolean 


If this attribute is TRUE, the media referenced by the Generate 
Media attribute will be used from this point on, whenever it is 
permissible to generate a track. If the Suppress attribute is 
FAI SF the actual veneration of the deoendent track will beein 
from the start time of the media item that is associated with 
these attributes. If the Suppress attribute is TRUE, generation 

mi/ill r"% /"vt 1*\c»*"rif» until o m if AtYI xiritV* tKo Cm v% wr\ hapc ottnKiltA C* 

win noi oegin uniu a media iiem wun ine suppress aiinouie set 
to FALSE is encountered. Generation using the referenced 
media continues until another media item with an Apply 
attribute set to TRUE is encountered, but it will be suppressed 
for media items with the Suppress attribute set to TRUE. 


Suppress 


Boolean 


If this attribute is TRUE, generation is suppressed from the Start 

Tim** of* tVif* aQQOfiatpH m**Hia it^m for thf* rfnration of tHf* mpHia 

1 111 i\* \J 1 111*-' addU^lnl^U Jll^Ula ILOltl, 1U1 lllC- UUlalJUll Ul lllV' IJlvUIo 

item. 


Generate 
Media. 


Media 
Reference 


This references media to be used as a template for the media 
item to be generated. The generated media item will consist of 
a transformation of this media suitably extended or shortened to 
the required length. By convention, a reference to NULL or 
non-existent media causes track generation to be a null 
operation where no media is generated. 
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Table 1. Track Control Attribute Definitions 

In the examples shown in Figs. A and B, the track control attributes conform to 
the aggregated approach, and form data fields within the EDL elements for the track 
media items. It is also noted that the track transformations referred to in the Generate 
5 Media entry in Table 1 can include trimming, looping, speeding up or slowing down 
video media items; omitting, selecting or duplicating still images; looping or selecting 
sub-sequences of audio and so on. The media used as a template for the dependent media 
track can be a media item or a series of media items. 

Fig, A. shows a definition of an EDL element 201 for a media item (not shown) 
10 the EDL element 201 having track control attributes 204 therein. The EDL element 201 
has a data field 204 that incorporates data fields 206-208 relating to the track control 
attributes for a dependent media item. If the duration of the source media item associated 
with the EDL element 201 is zero, then the media element is referred to as a 
"placeholder" since it can establish characteristics of a dependent media item without 
15 contributing anything to the source track. 

Fig. B shows instances of two EDL elements 301 and 306 associated with the 
first two media items of a source track (not shown). The EDL element 301 is associated 
with a source media item 303 (designated as SI), and the track control attributes 302 
incorporated into the EDL element 301 indicate that a dependent media item (not shown) 
20 referenced by the Generate Media attribute 304 will be loosely synchronised with the 
source media item 303. The EDL element 306 is associated with a source media item 305 
(designated as CI), but does not generate a dependent media item, because the Generate 
Media field 307 is NULL as indicated by a reference numeral 308. 

According to the first or aggregated arrangement used in Figs. A and B, all EDL 
25 elements for a source media track incorporate track control attribute fields. However, not 
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all these fields need play a part in generating, suppressing or modifying dependent tracks, 
this being determined by the value of their respective track control attributes as defined in 
Table 1. EDL elements which play some active role are depicted pictorially as SI, SI 
(see for example 303 in Fig. B). EDL elements which play no active role are depicted 
5 pictorially as CI, CJ (see for. example 305 in Fig. B). 

The aggregated method of association depicted in Figs. A and B allows track 
control attributes to remain associated with EDL elements (for media items) by default. 
The decoupled approach allows EDL elements for media items to be somewhat separated 
from the track control elements (i.e. the EDL elements defining the track control 

10 attributes). As noted, the data structures associated with the track control attributes in the 
decoupled arrangement are referred to as track control elements. The decoupling is not 
complete, as will be described in relation to Fig. C, as some track control attributes 
remain in the EDL element for the media item. In the decoupled arrangement part of the 
association between EDL elements for media items and track control attributes is 

15 established by adjacency. EDL elements for media items and track control elements are 
intermixed in a list that is referred to as a modified EDL (to distinguish it from a 
"normal" EDL which does not contain track control data structures/elements). Track 
control attributes in this decoupled arrangement refer to the EDL element following them 
as depicted by 404 in Fig. C which associates the track control element 403 with the EDL 

20 element 30 1\ 

Fig. C shows, using the decoupled approach, EDL elements 30T and 306' that 
correspond to the EDL elements 301 and 306 respectively in Fig. B. In the aggregated 
arrangement, track control attributes are present in each EDL element in the source EDL. 
In order to render the track control attributes of a given EDL element in the aggregated 
25 arrangement inoperative, the Boolean variables Suppress and Apply are given values of 
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FALSE, and the Generate Media variable is provided with a NULL reference (see for 
example the EDL element 306 and the field 308 in Fig. B). In the decoupled 
arrangement, the track control attributes can simply be omitted as is the case with the 
EDL element 306'. Accordingly, in this decoupled representation, absent attributes are 
5 equivalent to NULL or FALSE control attributes in the aggregated arrangement in Fig B. 

Having regard to the EDL element 301' in Fig. C it is evident that the track 
control attributes Apply and Generate Media have been removed and incorporated in a 
separate track control element 403 that contains the track control attributes 401. In this 
decoupled representation, absent attributes (ie the absence of a track control element 

10 associated with the EDL element 306*) are equivalent to null or false control attributes of 
Fig. B (as found, for example, in the EDL element 306). It is noted that variables 
Suppress and Next, (see 405 and 406 respectively in Fig. C) which are related to 
generation of dependent tracks, are still found in the EDL element 301 \ 

With this decoupled method a system, such as an editing arrangement 

15 incorporating the disclosed track generation technique, may control whether associations 
between source tracks and dependent tracks remain or are broken. It should be 
appreciated that the track control attributes and the association itself can be refactored in 
different ways without affecting the track generation technique. In other words, the 
decoupling of the track control attributes 401 from the media item attributes 402 can be 

20 performed in different ways without affecting the track generation technique. * 

Returning to Fig. 2, the EDL 902, which is associated with the source track 906, 
is processed by the track control software application running on the computer system 
100 whenever it becomes necessary to update the dependent media item 911. This 
processing needs to be performed before the dependent media item 911 is previewed, 

25 either on its own or as part of a composite production. In an editor that enables preview 
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of changes as soon as the changes are made, this source track processing is performed 
whenever the source track changes. 

In order to generate the dependent media item 911, the source EDL 902 is 
processed in clip display order as depicted by an arrow 915, in order to determine 
5 contiguous sequences of EDL elements which reference the same media, and thus define 
a range of the dependent generated media item 911. For every sequence of EDL elements 
generating a dependent media item, a portion of new dependent track, conforming to the 
duration and start time of the sequence of EDL elements, is generated using the media 
that is referenced by the relevant Generate Media attribute (see 710 in Fig. F). 

10 When track control attributes of an EDL element assign to the attribute Apply a 

value of TRUE, any previous media in use in the dependent track is replaced. Provided 
that the attributes of the EDL element does not specify that the generation should be 
skipped for this media item (ie Suppress = TRUE), generation of the dependent track 
using newly referenced media will commence from the start point of the current media 

15 item, otherwise the dependent track will begin from the start point of the first media item 
that permits overlaying (ie which permits generation of the dependent track). Generation 
of the dependent track will continue from that point on, being present for (ie synchronised 
with) all media items except those that have their Suppress attribute set to TRUE, until 
the generated media is either removed, replaced, or the end of the media item list (ie the 

20 EDL) for the source track is reached. 

The length of generated media item in the dependent track is determined by the 
length of sequences in the source EDL that permit generation. If the length of the media 
to be used for the dependent media item based on the Generate Media attribute does not 
conform to the duration of the source sequence it is being synchronised with, the 

25 dependent track is extended or shortened in duration. For example, if the dependent 
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media item would be too long in duration, it may be trimmed. If the dependent media 
item would be too short, it will be repeated or looped to attain the required length, with 
the last instance being trimmed. It should be appreciated that these approaches are 
examples only, and there are other ways to extend or shorten media that can be used. 
5 Fig. Da provides an example of generating a dependent track from a source 

track. A sequence of media items 502, 503, 519 ... of varying durations is depicted in 
increasing time order, depicted by an arrow 522, thereby forming a source track 501 . The 
EDL elements (not shown) for the media items in the track 501 have various track control 
attributes. The assigned values of these attributes are defined in Fig, E. From a 

10 terminology perspective, "X" means TRUE in Fig. E, and a blank entry means FALSE 
when considering the Suppress and Apply columns in Fig. E. Thus, for example, a 
media element 502 (designated as SI in the left hand column 601 of Fig. E) has 
associated track control attribute values Suppress = TRUE, Apply = TRUE, and Media 
Generation = Ol. Furthermore, a media element CI in the column 601 of Fig. E has 

15 associated track control attribute values Suppress = FALSE, Apply = FALSE, and 
Media Generation = NULL. 

Returning to Fig. Da a dependent media item 514 is generated by track control 
attributes in the EDL elements associated with the source track 50 1. The dependent track 
514 commences with a media item 504 with a designation "OF " indicating that the 

20 media item Ol ' is generated using a base media item Ol. The media item 504 (ie Ol ') 
may, however, be different in length from the base media item Ol. It is noted that both 
the dependent media item OF (ie 504) and a dependent media item 507 (designated as 
OF*) are generated from the media item Ol. A blank region 513 in the dependent track 
514 indicates that no media item is generated in the noted area 513. 
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Fig. Db shows an amended source track 508, which is the source track 501 after 
a number of changes have been made thereto. Some of the changes are set out as follows, 
(a) Media item 515 (ie S3 in the source track 501) and a media item 516 (ie C8 in the 
source track 501) have been moved to the start of the track 508. (b) A source media 
5 element 519 (ie C2 in the source track 501) has been deleted and thus does not appear in 
the amended source track 508. (c) The media element 505 (ie S2 in the source track 501) 
is replaced by a media element 520 (ie S4) in the source track 501). (d) An EDL for a 
media item 522 (ie C6 in the source track 501) is moved to the end of the EDL for the 
amended source track 508, and a media item 523 (ie C7 in the source track 501) is 

10 deleted, (e) A (new) media item 521 (ie S5 in the amended source track 508) is added. 
The final outcomes of the above manipulations are as follows. The dependent media item 
517 automatically travels, according to the disclosed track generation method, with the 
media item 515 (ie S3 in the source track 501) to form the dependent media item 510 (ie 
02" in a dependent track 518 that is loosely synchronised with the altered source track 

15 508). The duration of the dependent media item 510 is automatically adjusted according 
to corresponding track control attributes in the altered source track 508. A gap is caused 
in the dependent track 518 generated between the start of the media item 520 (ie S4) and 
the start of a media item 521 (ie S5 in the amended source track 508). A track of new 
material 521 (ie 03' in the dependent track 518) is generated. 03'" is generated between 

20 S 1 and S4. 

Considering Figs. Da, Db and E, with respect to overlays it is noted that the 
attributes of S2 (refer to the fifth entry in the column 601 in Fig. E) do not affect which 
overlays should be applied (since the attribute Generate Media = NULL) except that it 
does not permit itself to be overlaid (since the attribute Suppress = TRUE). S4 (ie the 
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second last entry in the column 601 in Fig. E) is in effect a null overlay, since it begins a 
region where overlays will not be applied. 

It is noted that the regions of overlays in the dependent track can be adjusted by 
operating on SI, S3, S4 alone. Each of SI, S2, S3, S4 act to define an end to regions that 
5 border on each other commence, by virtue that once an overlay is in effect it continues 
until overridden. Note also if CI, C2, C3 etc are deleted, copied or moved, the region of 
the respective overlay that encompasses them expands or contracts appropriately. Further 
note that this lack of explicit control allows the design decision of an author of automatic 
editing rules to continue to influence the results of the manual editing process. 

10 Fig. F shows a track generation process 700 in flowchart form. The acronym EE 

is used in Fig. F to indicate the term "EDL element". The process 700 commences with a 
step 701 that sets a current EDL element index to the first EDL element. Thereafter, a 
testing step 702 determines whether the index lies beyond the last EDL element. If this is 
the case, then the process 700 follows a "yes" arrow to a stop step 713. Otherwise, the 

15 process 700 follows a "no" arrow to a step 703 which determines if the current EDL 
element permits (continued) generation of a dependent EDL element. In other words, this 
step checks if the Suppress attribute is FALSE. If so, then the process 700 follows a 
"yes" arrow to a step 704 that that defines the starting index for generating the dependent 
media item to be the index of the current EDL element. If not, then the process 700 

20 follows a "no" arrow from the step 703 to a step 705 that defines the starting index for 
generating the dependent media item to be the index of the EDL element following the 
current EDL element. 

The process 700 is directed, from both the steps 704 and the step 705, to a step 
706 which sets the end index for the dependent media being generated to be the index of 

25 the next EDL element after the current element with (Apply = TRUE) OR (Suppress = 
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TRUE). Thereafter, a step 707 determines whether the current EDL element has the 
attribute Apply = TRUE. If so, then the process 700 follows a "y es " arrow to a step 708 
that saves the value of Generate Media. A following step 709 determines a media value 
previously saved by the step 708, on the present or a previous iteration and tests if there is 
5 at least one EDL element between the start and end indices (as determined in the steps 
704, 705). Returning to the step 707, if Apply NOT = TRUE then the process 700 
follows a "no" arrow to the step 709. 

If the stated condition for the step 709 is met, then the process 700 follows a 
"yes" arrow to a step 710 that generates a media item from the start time of the EDL 

10 element referenced by the start index time of the EDL element referenced by the end 
index, until the end of the dependent track sequence. Thereafter, a step 71 1 advances the 
current EDL element item index to the next media item with attribute values (Apply = 
TRUE) OR (Suppress = TRUE). Returning to the step 709, if the condition is not met, 
then the process 700 follows a "no" arrow to the step 711. After the step 711, the process 

15 700 follows an arrow 712 back to the step 702. 

A Graphical User Interface (GUI) that is applicable to the track generation 
method in some cases is now described. The extent to which the GUI needs to provide 
user interaction with the track control attributes varies with the editing system in which 
the track generation technique is incorporated. In some editors, there is little or no need 

20 for a GUI in regard to track control attributes. Thus, for example, in an editor that 
provides automatic editing facilities, the track control attributes can be set when 
automatic editing is applied, and the user need be given no further control over these 
attributes. In such a case, the automatic editing process can set the track control attributes 
as it considers appropriate. 
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For example, considering the source track 501 in Fig. Da, the auto-editor may 
automatically insert the clips SI, S2, S3, S4 and set the relevant track control attributes to 
use media Ol, 02 which are video overlays. If the user moves, copies, or deletes these 
clips, SI, S3, S4 overlays will be similarly moved, copied or deleted according to the 
5 disclosed track generation method. 

If desired, default track overlay media can also be defined to allow an overlay to 
be commenced before an EDL element applying an overlay is encountered. Note also 
that a user may concentrate on their main editing task without the need to consider 
overlays. 

10 Fig* G shows an example 800 of a GUI for a manual editing arrangements, in 

which the user may interact with the track control attributes. Implementation of this GUI 
is particularly simple if the decoupled EDL arrangement depicted in Fig. C is used, 
however the GUI can be used with the aggregated EDL arrangement as well. 

The GUI 800 provides representations 801, 802 of track control attributes on the 

15 same display where it depicts a story-board or timeline made up of media elements 803, 
804. ... The position of overlays can be adjusted by moving the indicators, 801 , 802. An 
indication that dependent tracks are suppressed is also made available by modifying the 
appearance of those clip where overlays are suppressed, as shown in the shading of the 
GUI element 805. In this arrangement, a users can reposition dependent tracks simply by 

20 altering the position of the indicators 801, 802 such as by dragging them with a mouse to 
a new position. The user can also decide where to position media items based on the 
position of the track control indicators 801, 802, and therefore determine which 
dependent track is to apply to which media item. The user can also control the Suppress 
attribute in clips. 

25 
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Industrial Applicability 

It is apparent from the above that the arrangements described are applicable to 
the data processing industries. 

The foregoing describes only some embodiments of the present invention, and 
modifications and/or changes can be made thereto without departing from the scope and 
spirit of the invention, the embodiments being illustrative and not restrictive. 
AUSTRALIA ONLY 

In the context of this specification, the word "comprising" means "including 
principally but not necessarily solely" or "having" or "including", and not "consisting 
only of. Variations of the word "comprising", such as "comprise" and "comprises" have 
correspondingly varied meanings. 
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The claims defining the invention are as follows: 

1. A method of generating, in relation to a source placeholder, a dependent media 
item, said method comprising the steps of: 

5 determining, from an EDL element associated with the source placeholder, a 

plurality of track control attributes; and 

establishing the dependent media item according to the track control attributes. 

2. A method according to claim 1, wherein the source placeholder is a source media 
10 item. 

3. A method according to claim 2, wherein the dependent media item is a graphical 
overlay. 

15 4. A method according to claim 3, wherein the plurality of track control attributes 

are one of included in the EDL element, and referenced by a dependent track reference in 
the EDL element. 

5. A method according to claim 3, wherein said plurality of track control attributes 
20 includes an activation attribute and a reference to another media item. 

6. A method according to claim 5, wherein the activation attribute comprises one of 
an apply attribute adapted to activate the dependent media item and a suppress attribute 
adapted to deactivate the dependent media item. 

25 
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7. A method according to claim 6, wherein activation of the dependent media item 

occurs at one of (a) the boundaries of the source media item, (b) between the boundaries 
of the source media item, and (c) outside the boundaries of the source media item. 

5 8. A method according to claim 5 wherein the other media item is a plurality of 

media items. 

9. An apparatus for implementing any one of the above methods. 

10 10. A computer program for directing a processor to perform any one of the above- 
noted methods. 

1 1 . Omnibus claims for the above-noted method, apparatus, and computer program. 

1 5 DATED this twenty-first Day of March, 2003 

Canon Information Systems Research Australia Pty Ltd 
Patent Attorneys for the Applicant 
SPRUSON&FERGUSON 
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